It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, in which other discretization schemes and/or adaptive time stepping techniques can be used to improve the performance of residual networks. Here, we propose \OURS, which extends this approach by introducing a framework that allows ODE-based evolution for both the weights and the activations, in a coupled formulation. Such an approach provides more modeling flexibility, and it can help with generalization performance. We present the formulation of \OURS, derive optimality conditions, and implement the coupled framework in PyTorch.

We thank the reviewers for their careful reading of the paper. Table 1: Neural ODE / JSDE predicted conditional intensity error. MAPE ODE JSDE Poisson 1.2 1.3 Hawkes (E) 172.0 5.9 Hawkes (PL) 91.4 17.1 Self-Correcting 27.2 9.3 expect the model to only work well for Poisson process which does not We will add the following details to the paper. Reviewer 1 also noted the Poisson dataset does not fit well to the Poisson process. We find that using longer Poisson sequences remedies this issue.

There exist numerous nonlinear partial differential equations in dispersive, as well as in dissipative systems which are of broad applicability in a wide range of sp atio-temporally dependent physical and biological settings. For instance, on the dispersive si de, the Nonlinear Schr odinger (NLS) model [1, 2] has been argued to be of relevance to optical [3, 4 ] and atomic physics [5, 6, 7] to plasma [8, 9] as well as fluid research [10, 9] and even to bio logical applications such as the DNA denaturation [11, 12]. This work has also been central to the seminal contributions of S. Aubry, especially in connection to discrete solitons and br eathers [13, 14, 15]. In a similar vein, in dissipative, reaction-diffusion systems one of the central m odels has been the Fisher-KPP (FKPP) equation which was originally conceived in the context of sp atial spread of advantageous alleles, and has since been employed to model species invasion, popul ation dispersal, and ecological front propagation [16, 17]. The FKPP model has also constituted fo r almost a century now a hallmark of reaction-diffusion models with a wide range of application s to cancer modeling, wound healing, flame propagation and a diverse further host of applications up to this day [18, 19] While such classical PDE models have for decades now been at t he center of initially analytical and subsequently computational studies, over the past deca de or so, a new suite of data-driven tools and techniques has met with explosive growth offering a n ew avenue and a fresh perspective enabling unprecedented developments.

It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, in which other discretization schemes and/or adaptive time stepping techniques can be used to improve the performance of residual networks. Here, we propose \OURS, which extends this approach by introducing a framework that allows ODE-based evolution for both the weights and the activations, in a coupled formulation. Such an approach provides more modeling flexibility, and it can help with generalization performance. We present the formulation of \OURS, derive optimality conditions, and implement the coupled framework in PyTorch.

The PDE-inspired formulation of coupled ODE is very interesting and can enable utilization of decades of progress in efficiently solving particular classes of coupled equations, in deep learning applications. This is a very exciting connection discovered by the authors. The central contribution of modeling weight evolution using ODEs hinges on the mentioned problem of neural ODEs exhibiting inaccuracy while recomputing activations. It appears a previous paper first reported this issue. The reviewer is not convinced about this problem.

It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, in which other discretization schemes and/or adaptive time stepping techniques can be used to improve the performance of residual networks. Here, we propose \OURS, which extends this approach by introducing a framework that allows ODE-based evolution for both the weights and the activations, in a coupled formulation. Such an approach provides more modeling flexibility, and it can help with generalization performance. We present the formulation of \OURS, derive optimality conditions, and implement the coupled framework in PyTorch.

It has been observed that residual networks can be viewed as the explicit Euler discretization of an Ordinary Differential Equation (ODE). This observation motivated the introduction of so-called Neural ODEs, in which other discretization schemes and/or adaptive time stepping techniques can be used to improve the performance of residual networks. Here, we propose \OURS, which extends this approach by introducing a framework that allows ODE-based evolution for both the weights and the activations, in a coupled formulation. Such an approach provides more modeling flexibility, and it can help with generalization performance. We present the formulation of \OURS, derive optimality conditions, and implement the coupled framework in PyTorch.